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REUSE METRICS AND MEASUREMENT - A FRAMEWORK r.D 

Donald J. Reifer, President 
Reifer Consultants, Inc. 

Torrance, CA 90505 

Abstract : This presentation will describe the lessons learned and experience 

gleaned by those firms which have started to Implement the reuse metrics and 
measurement framework prepared by the Joint Integrated Avionics Working Group 
(JIAWG) for use In controlling the development of common avionics and software 
for Its affiliated aircraft programs (e.g., the Air Force’s Advanced Tactical 
Fighter (ATF), the Army’s LH helicopter and the Navy’s A- 12 fighter). The 
framework was developed to permit the JIAWG and Service System Program Offices 
(SPOs) to measure the long-term cost/benefits resulting from the creation and 
use of Reusable Software Objects (RSOs). The framework also monitors the 
efficiency and effectiveness of the JIAWG’ s Software Reuse Library (SRL). 

The presentation will begin by defining the metrics and measurement framework 
which was established to allow the following six determinations and findings 
to be made relative to software reuse: 

1. Impact of RSO creation on software cost and productivity. 

2. Impact of RSO reuse on software cost and productivity. 

3. Impact of RSO mining on software cost and productivity. 

4. Minimum standards of quality for RSOs as they enter the SRL. 

5. Efficiency and effectiveness of SRL usage. 

6. Long-term cost/benefits of SRL usage. 

The presentation will discuss how the following seven criteria were used to 
guide the establishment of the proposed reuse framework: 

1. Compatible - The framework should be compatible with the software 
processes used by JIAWG contractors to develop avionics software 
products in Ada under DoD-STD-2167A. 

2. Ease of Data Collection - The data needed to quantify the metric 
should be easy to collect and normalize. 

3. Ease of Understanding - The metrics employed should be easy to 
understand, analyze and interpret. 

4. Minimum Cost - The measurement costs (i.e., data collection, 
analysis and reporting) should be kept to a minimum. 

5. Nonobtrusi ve - Collection of metrics data must not adversely 
impact the processes or products being measured. 

6. Objective - It should be difficult to bias or distort the value of 
the metric. 

7. Predictive - The metric should facilitate generation of accurate 
estimates of software cost, productivity and quality. 
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Next, object recapture and creation metrics will be explained along with their 
normalized use In effort, productivity and quality determination. A single 
and multiple reuse Instance version of the popular C0C0H0 cost model will be 
presented which employs these netrics and the measurement scheme proposed by 
the Software Productivity Consortium (SPC) to predict the software effort and 
duration under various reuse assumptions. Investigations in using this nodiel 
to predict actuals taken from the RCI database of over one thousand completed 
projects will be discussed along with statistical findings. 

User experience with this metrics and measurement framework as part of the Air 
Force’s Reusable Ada Avionics Software Package (RAASP) and Avionics Fault- 
Tolerant Software/ Ada Technology Insertion Program (AFTS/ATIP) projects will 
be discussed next. The lessons learned with these metrics by these projects 
will be siamurized. These two projects are conducting controlled experiments 
to capture measurement data that provides Insight irin those factors which 
Impact software cost, quality, productivity and system srforraance. The RAASP 
effort is focusing on determining the relative impact of object-oriented 
methods, reuse paradigms and SRL operational policies software productivity, 
cost and quality. AFTS/ATIP Is assessing the Impact of a large number of 
process and product factors on overall cost and system performance. 

The presentation will conclude with a summary of key points. Recommendations 
will be presented to help those embarking on a reuse program to improve their 
measurement and prediction capabilities. 
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PURPOSE 
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Describe the reuse metrics and measurement framework 
created by JIAWG to make the following determinations: 

• Impact of RSO acquisition on software cost and 
productivity 

• Impact of RSO reuse on software cost and 
productivity 

• Minimum standards of quality for RSOs entering the 
Software Reuse Library (SRL) 

• Efficiency and effectiveness of SRL usage 
. • Long-term cost/benefits of SRL usage 

Discuss implementation of the framework on the OSS and 
RAASP projects 
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Reusable Software Object fRSO) - life cycle products developed 
to be reused (designs, algorithms, code, tests/tost cases, etc.) 
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PRODUCTIVITY IMPROVEMENT STRATEGIES 
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* Productivity must be measured from a quality viewpoint 


M imIuM eepyiiyU fl(5l NoHa^« wlhoul fxkxwi l!*n eonwni 


I 





RCJ 

PH*taf 19 


s age3 


BARRIERS TO REUSE 
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• Lack of incentives 

• Few standards 

• Limited tool support 

• Champion needed 

• Multiple quality levels 

• NIH bias 

• Needed infrastructure changes 

• Few quantitative metrics 

Source : RCI Reuse Survey, 8/8P 
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METRICS SELECTION CRITERIA 
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• Compatible with DOD processes 

• Ease of data collection 

• Ease of understanding 

• Minimum measurement cost 

• Objective and unbiased 

• Predictive of the future 

• Unobtrusive as possible 
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OBJECT ACQUISITION RATIO 


OBJECT REUSE RATIO 


n 


n 


OAR = 2 (Wj) (aj/Aj) 


ORR = E (wj) (rj/Rj) 


1=1 


i=1 • 


where: aj = no. of RSOs acquired 


where: q = no. of reused objects 

i 

! 

per collection 


in a collection 

i 

Aj p no. of objects in that 


Rj = no. of objects in that 


collection 


collection 

i 

n - no. of collections 


n = no. of collections 

HE 
5 ’ 

Wj = weighting factor for 


Wj = weighting factor for 

each collection 

n 


each collection 

n 


and E Wj = 1; aj/Aj > 0 


and E wj = 1; q/Rj > 0 


i=1 


i=1 
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REUSE MECHANIZATION 
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Collection Wj 

• Requirements 0.20 

• Design 0.30 

• Source code 0.20 

• Tests/test cases 0.30 

• OAR 0 = (0.2)(a 1n + a^ r )/A^ + (0.3)(a2 n + a 3p + a 2r)^2 + 

(0.2)(a 3n + a 3p + a 3r )/A 3 + (0.3)(a 4n + a 4p + a 4( yA 4 

where: a xn = newly created objects 
a xp «=» purchased objects 
a xr = recovered objects 

• ORRq « (0.2) r |/R| + (0.3) ^ 21^2 + (0*2) r 3 /R 3 + (0.3) r 4 /R 4 

where: r x /R x = reuse ratio for a collection 
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METRICS USAGE 
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REUSE VERSION COCOMO (SINGLE INSTANCE) 




• Effort,. = c (I + (OAR x )(b 18 ) - (ORR 0 )(b 19 )) Effort 

where: c = adjustment factor for domain 

b 18 = RSO cost factor (0.10 < b 18 < 0.36) 
b ig » RSO benefits factor (0.20 < b 19 < 0.60) 

OAR x = expanded form of OAR 
ORR e » effective form of ORR 
Effort,. - cost in staff-months with reuse 
Effort - cost in staff-months (COCOMO) 

• OARx = (0.2)(a| n + (0.5) a lr )/A 1 + (0.3)(a 2n + (0.2) a 2p + 

(0.4) a 2r )/A 2 + (0.2)(a 8n + (0.2) a 8 p + (0.5) a 3r )/A 3 + 

(0.3)(a 4n + (0.3) a 4p + (0.0) a 4r )/A 4 
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FACTOR RATINGS 
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LOW 

NOMINAL 

HIGH 

VERY HIGH 

EXTRA HIGH 

Reuse 

Cost 

Factor 

( b 18) 

0.10 

0.17 

0.26 

0.31 

0.36 

Limited 

reuse 

packaging 

Design and 
code RSO 
reuse 
packaging 

Full RSO 
reuse 
packaging 

Domain 
specific RSO 
reuse 
packaging 

Extensive 

reuse 

packaging 

(synthesis) 


Reuse 
* Benefits 
2 « Factor 
5 (big) 

I 


LOW 

NOMINAL 

HIGH 

VERY HIGH 

EXTRA HIGH 

0.20 

0.25 

0.34 

0.48 

0.60 

Planned 

reuse 

Systematic 

reuse 

Managed 

reuse 

Institutionalized 

reuse 

(within and 
across jobs) 

Optimized 

reuse 

(domain 

specific) 
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METRICS USAGE 
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SPC MODEL 


Cost = (1-R) + R(B + EM) 

where: B « relative cost to reuse RSO 
R - proportion of reused software 
E « cost to develop RSO 
N = number of reuses 


REUSE COCOMO MODEL (MULTIPLE INSTANCES) 


Effort m = c(1 , (OAR x ) (b 1fi ) - (ORR e ) (b 19 )) Effort 

m 

where: c = calibration coefficient 

m « number of reuses (m > 1) 
b 18 = cost factor (0.10 < b^ < 0.36) 

b^ = benefit factor (0.20 < b^ < 0.60) 

OAR x - Object Acquisition Ratio (average) 

ORR e - Object Reuse Ratio (average) 

AlmaUriab copyright by RCI. llollobaiepioducadwKhoulpoocwiinanconsant. 
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THE OSS EXAMPLE 
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QUALITY METRICS 
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FACTOR 


CRITERIA 


METRICS 


• Correctness 

• Efficiency 

• Maintainability 

• Portability 

• Testability 

• Usability 


>> Clarity 

>- Coupling Strength 
>■ Independence - 
■> Modularity 

Self-descriptiveness 
>■ Simplicity 
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iojndep task-indep machjndep softjndep physjimjndep 


I softjndep = no_sys_dep_mod + no_impl_dep _pragmas I 
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Cumulative number 
of times SRL browsed 
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3RARY EFFICIENCY 
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Average service time 
System response time 
System throughput 
Resource utilization 
Workload characteristics 


Cumulative number 
of times RSO retrieved 


Cumulative number 
of times SRL searched 
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I FACTOR | 

• Efficiency 

• Effectiveness 


Number of 
s } users/month 
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LONG-TERM COST/BENEFITS 
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NPW = £ (B t ) (1/(1 +i)‘) 


1 

t=0 

i 

NON-RECURRING COSTS 

TANGIBLE BENEFITS 

• Acquisition $ 

• Adaptation 

• Documentation 

• Infrastructure 

• Training 

• Cost avoidance $ 

• Added capability 

• Reduced cost 

of quality 

• Cost savings 

COSTS $ 

BENEFITS $ 



• Administration $ 

• Maintenance 

• Operations 

• Better customer 

satisfaction $ 

• Fitness for use 

COSTS $ 

BENEFITS $ 
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RAASP ADAPTATION 
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HYPERTEXT LIBRARY EFFICIENCY 
AND EFFECTIVENESS 


• No. of objects In library 

• No. of links traversed/hit 

• No. of items browsed/hit 

• Amount of time for a hit 

• No. of log in’s per user 

• Amount of time/user session 

• No. of objects withdrawn/user session 

• No. librarian actions/object 

• No. of objects submifted/month 

• No. of objects withdrawn/month 

• No. of SPRs/object/month 

• No. of SCRs/object/month 



Usage Profiles 

• By object 

• By service 

• System-wide 
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SUMMARY AND CONCLUSIONS 
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• We've described the JIAWQ software reuse metrics and 
measurement framework 

• We've described the pilot implementation of the framework 
on OSS and RAASP 

• We've discussed our multiple instance reuse version of the 
COCOMO model 

• Needed to explore the economics of reuse 

• We've just touched the surface of the issues involved 

• Your thoughts, feedback and help are solicited especially if 
you have "hard" data to share 
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